Many different types of control and interface devices have been developed for inputting commands into a machine. For example, hand-manipulated interface devices, such computer mice, joysticks, trackballs, and keyboards, commonly are used to input instructions into a computer by manipulating the interface device. Such interface devices allow a user to control movement of a virtual pointer, such as a cursor, across a computer screen, select or move an icon or other virtual object displayed on the computer screen, and open and close menu items corresponding to different input commands.
Recently, video image based machine interfacing systems are being developed. Many of these interfacing systems are based on the detection of an input command in images captured by a single video camera. In one approach, the light-emitting element of a pen light is used to control the displayed position of a virtual pointer on a computer screen. At least one input button, such as a button on the light pen or a key on a computer keyboard, typically is used in this approach to perform the functions of the corresponding buttons on a computer mouse. In another approach, a video camera is mounted above the keyboard portion of the computer to capture images of a computer user's hand that is placed within a selected observation zone. Cursor control circuitry monitors the presence, configuration and movement of the user's hand based on the captured images. When the user's hand is placed in a first predetermined configuration and moved into and through the observation zone, the cursor control circuitry switches the computer from a keyboard typing mode to a cursor positioning mode, tracks movement of the hand through the observation zone, and correspondingly moves the cursor on the computer display screen. When the user's hand is placed in a predetermined second configuration, the cursor control circuitry causes a cursor “pick” function to be generated in conjunction with the repositioned cursor. In response to placement of the user's hand in a third predetermined configuration, the circuitry switches the computer from its cursor positioning mode back to its previous keyboard typing mode.
Other hand gesture based video image machine interfacing systems have been proposed. One approach analyzes human hand and finger motions in images captured by a video camera. These motions are interpreted as user input operations on a virtual input device. The resulting optical user-computer interface complements the visual screen-to-user path with a hand-to-image data acquisition link from the user to the processor unit. The interface allows different types of manually-operated input devices (mouse, keyboard, etc.) to be emulated.
Another hand gesture based video image machine interfacing approach uses images of hand gestures to control a computer. In this approach, motion of the user's hand is tracked in a three-dimensional coordinate system with five degrees of freedom. The system includes an image processor and at least two cameras. During operation of the system, hand images received from the cameras are processed to recognize predefined hand gestures. The computer utilizes two-dimensional images obtained by each camera to derive three-dimensional position and orientation coordinates of the extended finger. The recognized hand gestures are interpreted as commands, which are sent to an application being executed by the computer.